Accelerating recommendation system training by leveraging popular choices

Recommender models are commonly used to suggest relevant items a user for e-commerce and online advertisement-based applications. These use massive embedding tables store numerical representation of items' users' categorical variables (memory intensive) employ neural networks (compute generate final recommendations. Training these large-scale recommendation is evolving require increasing data compute resources. The highly parallel portion can benefit from GPU acceleration however, large often cannot fit in the limited-capacity device memory. Hence, this paper deep dives into semantics training obtains insights about feature access, transfer, usage patterns models. We observe that, due popularity certain inputs, accesses embeddings skewed with few entries being accessed up 10000X more. This leverages asymmetrical access pattern offer framework, called FAE, proposes hot-embedding aware layout recommender utilizes scarce memory storing embeddings, thus reduces transfers CPU GPU. At same time, FAE engages accelerate executions hot entries. Experiments on production-scale real datasets show that overall time by 2.3X 1.52X comparison XDL CPU-only CPU-GPU execution while maintaining baseline accuracy.